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Abstract:  A  plaque  assay  for  quantitating  filoviruses  in  virus  stocks,  prepared  viral  challenge 
inocula  and  samples  from  research  animals  has  recently  been  fully  characterized  and  standardized 
for  use  across  multiple  institutions  performing  BSL-4  studies.  After  standardization  studies  were 
completed.  Good  Laboratory  Practices  (GLP)-compliant  plaque  assay  method  validation  studies  to 
demonstrate  suitability  for  reliable  and  reproducible  measurement  of  Marburg  Virus  Angola 
(MARV)  variant  and  Ebola  virus  Kikwit  (EBOV)  variant  commenced  at  USAMRIID.  Validation 
parameters  tested  included  accuracy,  precision,  linearity,  robustness,  stability  of  the  virus  stocks 
and  system  suitability.  The  MARV  and  EBOV  assays  were  confirmed  to  be  accurate  to  ±  0.5  logio 
PFU/mL.  Repeatability  precision,  intermediate  precision,  and  reproducibility  precision  were 
sufficient  to  return  viral  titers  with  a  %CV  of  <  30%,  deemed  acceptable  variation  for  a  cell-based 
bioassay.  Intraclass  correlation  statistical  techniques  for  evaluation  of  the  assay's  precision  when 
the  same  plaques  were  quantitated  by  2  analysts  returned  values  passing  acceptance  criteria, 
indicating  high  agreement  between  analysts.  The  assay  was  shown  to  be  accurate  and  specific 
when  run  on  NHP  serum  and  plasma  samples  diluted  in  plaque  assay  medium,  with  negligible 
matrix  effects.  Virus  stocks  demonstrated  stability  for  freeze-thaw  cycles  typical  of  normal  usage 
during  assay  retests.  The  results  demonstrated  that  the  EBOV  and  MARV  plaque  assays  are 
accurate,  precise  and  robust  for  filovirus  titration  in  samples  associated  with  the  performance  of 
GLP  animal  model  studies. 

Keywords:  Plaque  assay;  filovirus;  Ebola;  ebolavirus;  marburgvirus;  Marburg  virus;  Vero  E6  cells; 
GLP  compliant;  validation;  animal  rule 
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1.  Introduction 

Filoviruses  are  zoonotic  viruses  belonging  to  the  virus  family  Filoviridae  and  can  cause  severe 
hemorrhagic  fever  in  humans  and  nonhuman  primafes  (NHP),  wifh  high  case  fafalify  rafes.  These 
viruses,  specifically  Ebola  (EBOV)  and  Marburg  (MARV)  viruses,  are  some  of  fhe  mosf  lefhal 
viruses  known  fo  man.  Filoviruses  can  be  defecfed  and  quantified  using  a  variety  of  basic  or 
sophisticated  virological  methods.  These  methods  include  but  are  not  limited  to  plaque  assay, 
reverse-transcription  polymerase  chain  reaction  (RT-PCR),  deep  sequencing,  transmission  electron 
microscopy  (TEM),  50%  tissue  culture  infectious  dose  assay  (TCIDso)  and  ViroCyt®  flow-based 
methods,  each  with  their  own  limitations  for  detection  and  quantitation  of  virus  genomic  material, 
viral  proteins  or  intact  infectious  particles  [1-4]. 

The  viral  plaque  assay  is  a  widely  used  technique  for  virus  isolation  and  purification,  and  the 
quantitation  of  infectious  viral  particles  within  a  sample  [5].  The  basis  of  the  technique  is  to  measure 
the  ability  of  a  plaque-forming  unit  (PFU)  of  virus  to  form  a  "plaque"  on  a  confluent  monolayer 
culture  of  adherent  cells.  A  plaque  results  from  infection  of  a  cell  by  a  plaque-forming  unit  followed 
by  the  replication  of  that  virus,  and  eventually,  the  death  of  the  cell.  From  that  cell,  the  newly 
replicated  virus  particles  infect  and  kill  surrounding  cells.  The  culture  is  then  stained  with  a  dye, 
discerning  the  cells  in  the  plaque  from  the  healthy  surrounding  monolayer  due  to  the  properties  of 
the  stain.  The  stain  can  be  a  vital  dye,  a  protein  stain,  or  a  virus-specific  stain  such  as  a  coupled 
antibody.  The  vital  dye  neutral  red  plaque  assay  has  been  in  use  at  USAMRIID  for  many  years  to 
enumerate  EBOV  and  MARV  viruses,  in  stock  solutions  and  prepared  challenge  material.  The 
assay  is  also  frequently  used  to  measure  Filovirus  samples  from  infected  NHP  in  support  of  animal 
model  development  and  countermeasures  testing  projects  at  USAMRIID.  The  reliable  and  accurate 
measurement  of  virus  in  stock  solutions,  challenge  material  and  NHP  blood  samples  is  an  essential 
activity  supporting  animal  model  studies  and  regulated  studies  for  the  advanced  development  of 
medical  countermeasures  (MCM)  under  Good  Laboratory  Practice-compliant  (GLP)  research 
conditions  [6] .  Due  to  its  use  in  support  of  regulated  studies,  this  plaque  assay  must  be  validated 
for  its  intended  use  [7] . 

The  purpose  of  these  EBOV  and  MARV  validation  studies  was  to  demonstrate  that  the  plaque 
assay  was  suitable  for  reliable  and  reproducible  measurement  of  infectious  EBOV  and  MARV  in 
virus  stock  solutions,  which  are  routinely  diluted  to  prepare  challenge  solutions.  This  study  did  not 
include  validation  of  quantitation  of  EBOV  virus  in  test  samples,  such  as  serum  and  plasma  from 
species  of  infected  NHP;  this  was  only  completed  for  MARV.  The  study  tested  cell  passage  ages  of 
the  Vero  E6  cells  that  are  currently  in  use  at  USAMRIID.  Based  on  the  nature  and  intended 
application  of  the  plaque  assay,  the  EBOV  and  MARV  plaque  assays  were  validated  for  the 
following  parameters:  accuracy,  precision,  linearity,  limit  of  detection  (LOD),  lower  limit  of 
quantification  (LLOQ),  robustness,  stability,  and  system  suitability.  For  precision  studies,  two 
analysts  performed  the  neutral  red  plaque  assay  in  parallel  on  two  separate  days,  and  collected 
their  own  data  as  well  as  re-counted  each  other's  plates.  Precision  studies  resulted  in  a  very  high 
degree  of  similarity  in  the  data  generated  by  two  analysts. 

2.  Materials  and  Methods 

2.1.  Virus.  A  USAMRIID  cell  culture  stock  of  Ebola  virus  (EBOV,  isolate  GDG#9510621)  originated 
from  an  outbreak  in  Kikwit,  Zaire  in  1995,  and  was  passaged  twice  at  the  Genters  for  Disease 
Gontrol  and  Prevention  (Atlanta,  GA,  USA),  and  once  in  Vero  E6  cells  at  USAMRIID  for  a  total  of  3 
passages.  This  stock  was  named  R4370,  and  it  was  used  to  prepare  the  positive  control  (PG)  and 
high,  medium  and  low  titer  quality  control  samples  (QGl,  QG2,  and  QG3,  respectively)  which  were 
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used  exclusively  as  the  test  samples  in  these  studies.  The  nominal  titers  of  these  stocks  are 
presented  in  Table  1. 

A  USAMRIID  cell  culture  stock  of  an  Angolan  isolafe  of  Marburg  virus  (MARV,  isolafe 
CDC#200501379),  originafed  from  an  oufbreak  in  Angola  in  2005,  and  was  passaged  4  fimes  in  Vero 
E6  cells.  This  stock  was  named  AIMS  17214,  and  if  was  used  fo  produce  fhe  MARV  PC,  QCl,  QC2 
and  QC3  samples  used  on  fhe  MARV  validafion  sfudy  (Table  2).  The  plaque  assay  negative  control 
(NC)  was  always  the  plaque  assay  medium  diluent  (IXMEM  with  5%  FBS)  into  which  the  PC  and 
QC  samples  were  diluted  for  each  experimenf.  Plated  NC  never  resulfed  in  any  plaques  for  any 
experimental  run. 

2.2.  Cells.  Vero  E6  cells  (C1008)  were  obtained  from  BEI  Resources  af  passage  number  24  (Manassas, 
VA,  USA).  BEI  Vero  E6  cells  are  Catalog  number  NR-596,  lots  3956812  and  3956593.  Passage 
numbers  presented  in  the  results  section  denote  cumulative  passage  numbers  upon  receipt  plus 
subsequent  passages  at  USAMRIID.  Cells  were  cultured  under  CLP  conditions,  where  all  media, 
supplements  and  procedures  for  passaging  and  culfuring  cells  were  performed  and  recorded  by 
CLP  frained  fechnicians  in  fhe  USAMRIID  Cell  Culfure  Laboratory.  Following  cell  culfuring, 
counting  and  plating  standard  operating  procedures  (SOPs)  at  USAMRIID,  cells  were  counted  by 
use  of  eifher  a  Cedex  HiRes  Mulfi  Sampler  System  (Roche,  Indianapolis,  IN,  USA),  a 
NucleoCounfer  NC200  (Chemomefec,  Davis,  CA,  USA)  or  a  hemocyfomefer.  Cells  were  plated  in  6- 
well  fissue  culfure  dishes  (Cafalog  no.  3506,  CoSfar,  Coming  Inc.  Coming,  NY,  USA)  or  100  mm 
tissue  culture  dishes  (Catalog  no.  430293,  Corning  Inc.,  Coming,  NY,  USA)  at  700,000  ±  25,000 
cells/well  of  a  6-well  plate,  or  4  x  lO'’  ±  150,000  cells/dish  in  10  mLs  of  medium  for  fhe  larger  dishes. 

2.3.  Reagents,  cell  culture  media,  supplements  and  stains.  Filovims  negafive,  normal  cynomolgus 

macaque  and  rhesus  macaque  serum  and  plasma  (KsEDTA)  were  purchased  from  Bioreclamafion 
(Wesfbury,  NY).  Reagenfs  for  culfuring  of  cells:  lx  MEM  (Caf.  No.  10010249,  Cellgro,  Manassas,  VA, 
USA)  in  plates,  supplemented  wifh  5  fo  10%  FBS  (vol/vol)  (Caf.  No.  16000-044,  Gibco,  Life 
Technologies,  Grand  Island,  NY,  USA),  5  mM  L-glufamine  (Gaf.  No.  SH330034.01,  Thermo  Scienfific 
HyGlone,  GE  Healfhcare,  Logan,  UT,  USA).  Reagenfs  for  plaque  assay:  2xEBME  (labeled  as  BME, 
Gaf.  No.  A15950DK,  Gibco,  Life  Technologies,  Grand  Island,  NY,  USA),  FBS  (Gaf.  No.  SH30071.03 
or  SH30071.02,  Thermo  Scientific  HyGlone,  GE  Healfhcare,  Logan,  UT,  USA),  penicillin- 
sfrepfomycin  (Gaf.  No. 

30-022-GI,  Gellgro,  Manassas,  VA,  USA),  SeaKem  ME  Agarose  (Gaf.  No.  50014,  Lonza,  Rockland, 
ME,  USA),  Neufral  red  from  Gibco  (Gaf.  No.  02-0066DG,  Gibco,  Life  Technologies,  Grand  Island, 
NY,  USA). 

2.4.  GLP  Compliance.  These  filovims  plaque  assay  validafion  sfudies  were  conducted  af  USAMRIID 
in  compliance  wifh  fhe  U.S.  Food  and  Drug  Adminisfrafion  Good  Laboratory  Pracfice  (GLP) 
Regulafions,  21  GFR  Parf  58.  Each  validation  was  conducted  according  to  a  GLP  study  protocol, 
with  any  GLP  exceptions,  SOP  or  protocol  deviations  identified  and  listed  in  fhe  profocol  and/or 
reporfs.  The  sfudies  were  inspected  and  audited  by  Qualify  Assurance  personnel  af  BSL-2  and  BSL- 
4  crifical  performance  phases.  Sfudy  profocols,  all  collected  dafa  and  final  reporfs  were  audifed  and 
archived  in  compliance  wifh  GLP  regulations. 

2.5.  Filovims  Plaque  Assay.  The  assay  was  performed  in  accordance  wifh  published  mefhods  [8]. 
Gells  were  plated  in  6-well  plafes  24  fo  72  hours  prior  fo  use  in  assay,  as  required  based  on 
experimental  design,  and  plates  were  carried  into  the  BSL-4  lab  at  time  of  use.  Agarose  was 
prepared  af  a  concenfrafion  of  1.0  g  agarose  per  100  mL  disfilled  wafer  and  aufoclaved  for  sterility. 
Prepared  sterilized  agarose  was  re-melted  by  microwave  at  BSL-4  prior  to  each  use.  QG  and  PG 
samples  were  diluted  in  lx  MEM,  supplemented  with  5%  FBS  (vol/vol),  in  a  10-fold  fashion 
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according  to  the  experimental  design.  The  culture  medium  in  the  6-well  plate  was  removed  by  swift 
decanting.  Virus  inocula  at  various  dilutions  were  added  to  duplicate  wells  of  6-well  plafes  in 
volumes  of  100  pL  or  200  pi,  or  1  mL  volumes  info  100  mm  culfure  dishes.  Inoculum  fluid  was 
disfribufed  by  genfle  manual  rocking.  The  plafes  were  incubafed  af  37  °C  for  1  hour  ±10  minufes, 
rocking  every  15  ±  5  minufes.  A  primary  liquid  overlay  confaining  a  1:1  rafio  of  2x  EBME  medium 
(supplemenfed  wifh  10%  FBS,  400  lU  penicillin,  400  pg/mL  sfrepfomycin)  equilibrafed  fo  37-42  °C 
and  1%  melfed  agarose  equilibrafed  fo  60°C  degrees  was  overlaid  in  2  ml  aliquofs  onfo  fhe 
inoculafed  monolayer  and  allowed  fo  solidify.  The  inoculum  was  nof  removed  from  fhe  wells  prior 
to  the  addition  of  overlay.  Plafes  were  incubafed  for  approximafely  7  days.  Af  Day  7  fhey  were 
sfained  wifh  2  ml  of  secondary  overlay  which  consisfed  of  1:1  volume  rafio  of  2x  EBME  plus 
supplemenfs:  1%  agarose  and  fhe  combined  volumes  were  supplemenfed  wifh  4%  neufral  red. 
Plaques  were  counfed  and  recorded  24-48  hours  posf-sfaining.  All  work  wifh  filoviruses  was 
carried  ouf  under  maximum  confainmenf  in  fhe  biosafety  level  4  laboratories  at  USAMRIID. 

2.6.  Plaque  counting  and  titer  calculation.  Plates  were  inverted  on  a  light  box  and  plaques  were 
counted  by  hand  with  the  aid  of  a  marker.  For  precision  experimenfs,  plafes  were  counfed  once  by 
fhe  performing  analysf,  fhen  ink  marks  were  genfly  removed  by  efhanol  wipe.  The  plafes  were  fhen 
recounted  by  a  second  analyst.  For  experiments  requiring  recounting  on  the  following  day,  ink 
marks  were  removed  and  fhe  plafes  were  placed  back  info  fhe  incubator  unfil  fhe  nexf  counfing. 
The  SOP  mefhod  dicfafed  fhaf  plaque  numbers  befween  10  and  150  would  be  counfed  and 
recorded,  and  used  in  calculafions  fo  defermine  quanfifafive  fifer  measuremenfs.  Plaque  numbers 
below  10  were  counfed  and  recorded  for  LOD  analyses  [5,9].  If  fhere  were  wells  af  fwo  differenf 
serial  dilufions  which  produced  counfs  wifhin  fhis  range,  fhen  fhe  counfs  from  wells  wifh  fhe  more 
concenfrafed  dilufion  were  chosen  for  calculafing  fifer.  Counts  from  replicafe  wells  were  averaged, 
and  fhe  average  was  multiplied  by  the  dilution  factor  of  fhe  inoculum  which  produced  fhaf 
number,  and  fhe  volume  of  inoculum  plafed  fo  calculafe  fhe  plaque  forming  unifs  (PFU)  per  ml  of 
fhe  original  stock  virus  preparafion.  The  calculafion  is:  average  value  of  plaques  in  replicafe  wells  x 
dilufion  factor  h-  virus  inoculum  volume  (in  mL,  e.g.  0.1  mL)  =  tifer  in  PFU/mL. 

2.7.  Variables  under  test  for  determination  of  assay  robustness.  Variables  such  as  passage  age  of  cells, 
amount  of  time  (in  days)  cells  can  be  seeded  in  plates  before  use,  freeze-fhaw  sfabilify  of  virus 
samples  in  cell  culfure  media,  nonhuman  primate  serum  and  plasma  samples  (for  MARV  only), 
amount  of  virus  inoculum  volume  and  tissue  culture  vessel,  and  optimal  days  upon  which  to  stain 
and  count  plaques  were  all  assessed  as  part  of  assay  robusfness  in  fhe  validafion  experimenfs. 
Specific  details  of  fhe  experimenfal  design  evaluafing  robusfness  are  presented  wifh  the  results  for 
fhaf  paramefer.  As  experimenfs  progressed  over  fime,  fhe  sysfem  suifabilify  was  defermined  by 
analyzing  fhe  "sfabilify"  of  the  cells,  or  effect  of  cell  age  (passage  number)  on  plaque  number  and 
qualify,  and  compared  across  all  experimenfs,  as  appropriafe. 

2.8.  Statistical  tests.  Raw  dafa  values  were  analyzed  and  described  wifh  basic  descripfive  sfafisfics 
for  presenfafion  as  mean  ±  sfandard  deviafion.  Comparisons  of  sample  groups  were  done  assuming 
equal  variance  using  fwo-failed  Sfudenf's  f-fesf  in  Microsoff  Excel  or  GraphPad  Prism  soffware  [10]. 
Coefficienfs  of  variafion  (%  CV)  were  calculafed  and  are  presented,  where  relevanf  [11]. 

3.  Results 

3.1.  Confirmation  of  Accuracy 

The  true  concentration  of  virus  fifer  can  be  measured  by  an  assay  fhaf  evaluates  fhe  amounf  of 
live,  replicafing  virus  in  a  sample.  To  date,  fhe  plaque  assay  is  fhe  mosf  reliable  mefhod  fhaf 
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indicates  the  true  titer,  in  plaque-forming  units  (PFU)  per  mL  of  sample.  The  validafion  profocol 
esfablished  fhe  fesf-specific  accepfance  criferia  for  accuracy  as:  8  ouf  of  af  leasf  10  buf  up  fo  12 
observed  filers  for  QCl,  QC2  and  QC3  musf  be  wifhin  ±  0.5  logio  of  fheir  nominal  filers,  and  fhe 
posifive  control  (PC)  sample  must  be  measured  within  ±  0.5  logio  of  ifs  nominal  filer  for  fhe  assay  fo 
pass  performance  criferia.  The  designafed  accuracy  experimenfs  for  MARV  and  EBOV  fesfed  fhis 
explicifly.  For  fhe  accuracy  fesfs  performed  wifh  each  virus,  measuremenfs  of  average  filers,  %CV, 
and  %  difference  from  nominal  filer  were  made  and  are  presenfed  in  Tables  1  and  2. 

The  %  difference  of  fhe  observed  filer  from  fhe  nominal  filer  is  an  imporfanf  measuremenf  when 
considering  accuracy.  The  average  observed  value  will  be  somewhaf  differenl  from  fhe  nominal 
tiler,  but  it  is  important  to  establish  an  acceptable  %  difference  based  on  how  fhe  assay  is  known  fo 
perform,  and  ifs  nafural  variabilify.  The  filers  for  MARV  and  EBOV  QCl,  QC2  and  QC3  are  all 
required  fo  fall  within  ±  0.5  logio  of  their  nominal  titers,  and  that  range  provides  the  values  from 
which  the  acceptable  %  differences  can  be  calculafed.  For  example:  If  fhe  low  end  of  fhe  accepfable 
tiler  range  is  1.018  x  10®  PFU/mL,  and  the  nominal  titer  is  3.52  x  10®  PFU/mL,  then  the  %  difference 
is  calculated  as  follows: 

(1.018x10® -3.52x10®!  x  100%  =  -71.1% 

3.52  X  10® 


Table  1.  EBOV  Accuracy  resulfs  in  fhe  confexf  of  fhe  ranges  of  accepfable  low  and  high  % 
differences  for  bofh  sefs  of  QCl,  QC2  and  QC3 


Sample 

Nominal 

Titer 

Low  End  of 
Range 
(PFU/Ml) 

[this  number  is  - 
0.5  logio] 

High  End  of 
Range 
(PFU/Ml) 

[this  number  is 
+0.5  logio] 

%  Difference 

of  Low  End 

From 

Nominal 

Titer 

%  Difference 
of  High  End 
From 

Nominal 

Titer 

QCl 

1.17  X  10® 

3.70  X  10® 

3.70  X  10® 

-68.4% 

+216% 

EBOV  QCl  average  observed  filer  (n=12)  was  6.18  xlO®  PFU/mL  (CV=3.9%). 

This  is  -47.2%  differenl  from  nominal,  and  fhe  value  passes. 

QC2 

1.07  X  10® 

3.38  X  10* 

3.38  X  10® 

-68.4% 

+216% 

EBOV  QC2  average  observed  filer  (n=12)  was  2.55x10®  PFU/mL  (CV=16.2%). 

This  is  a  -34.7%  difference  from  nominal,  and  fhe  value  passes. 

QC3 

1.06  X  10* 

3.35  X  10® 

3.35  X  10* 

-68.4% 

+216% 

EBOV  QC3  average  observed  filer  (n=12)  was  3.42  xlO*  PFU/mL  (CV=11.0%). 

This  is  a  -44.7%  difference  from  nominal,  and  fhe  value  passes. 

For  fhis  assay  run,  EBOV  PC  value  was  6.60  x  10*  PFU/mL,  which  falls  in  fhe  required  range  of  3.45 
X  10*  (-68.4%)  and  3.45  x  10®  PFU/mL  (+216%)  fo  pass. 


Table  2.  MARV  Accuracy  resulfs  in  fhe  contexf  of  fhe  ranges  of  accepfable  low  and  high  % 
differences  for  bofh  sefs  of  QCl,  QC2  and  QC3. 


Sample 

Nominal 

Titer 

Low  End  of 
Range 
(PFU/Ml) 

[this  number  is  - 
0.5  logio] 

High  End  of 
Range 
(PFU/Ml) 

[this  number  is 
+0.5  logio] 

%  Difference 

of  Low  End 

From 

Nominal 

Titer 

%  Difference 
of  High  End 
From 

Nominal 

Titer 

QCl 

3.5  X  10® 

1.096x10® 

1.096x10® 

-68.7% 

+213% 
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MARV  QCl  average  observed  titer  (n=12)  was  2.88  xlO®  PFU/mL  (CV=34.9%). 
This  is  -17.7%  different  from  nominal,  and  fhe  value  passes. 


QC2 

3.5  X  10® 

1.096x10® 

1.096x10® 

-68.7% 

+213% 

MARV  QC2  average  observed  titer  (n=12)  was  2.55x10®  PFU/mL  (CV=23.6%). 

This  is  a 

-6.4%  difference  from  nominal,  and  fhe  value  passes. 

QC3 

3.5  X  104 

1.096x104 

1.096x10® 

-68.7% 

+213% 

MARV  QC3  average  observed  filer  (n=12)  was  3.42  xlO*  PFU/mL  (CV=  20.1%). 
This  is  a  41.4%  difference  from  nominal,  and  fhe  value  passes. 


For  fhis  assay  run,  MARV  PC  value  was  4.5  x  10*’ PFU/mL,  which  falls  in  fhe  required  range  of  1.096 
X  10®  (-68.7%)  fo  1.096  x  10^ PFU/mL  (+218%)  fo  pass. 

Mosf  of  fhe  ofher  fesfs  on  fhe  validafion  profocol  were  designed  similarly  fo  fhe  Accuracy  fesf, 
with  repeated  testing  of  QCl,  QC2  and  QC3  and  comparison  of  fhose  resulfs  againsf  fhe  nominal 
tilers  for  EBOV  and  MARV  QCs.  The  EBOV  validation  had  9  additional  tests  with  this  same 
experimental  design  enabling  cross-comparison,  while  the  MARV  validation  had  another  8  tests. 
Passage  of  fhe  accuracy  criferia  across  fhose  fesfs  is  presenfed  in  Table  3. 


Table  3.  Tesfs  on  fhe  validafion  which  were  of  a  similar  design  to  the  accuracy  tests  and  passed  the 
accuracy  criteria 


Test 

Number  of  EBOV  Assay  Runs  Within 
Nominal  Titer  Range  for  Each  QC 
(%  difference  of  average  observed  titer 
from  nominal) 

Number  of  MARV  Assay  Runs  Within 
Nominal  Titer  Range  for  Each  QC 
(%  difference  of  average  observed  titer 
from  nominal) 

QCl  12  of  12  (-18.1%) 

QCl  12  of  12  (86.2%) 

Precision  1 

QC2  11  of  12  (6.3%) 

QC2  12  of  12  (105.8%) 

QC3  12  of  12  (3.3%) 

QC3  10  of  12  (175.7%) 

QCl  12  of  12  (105.5%) 

QCl  12  of  12  (98.8%) 

Precision  2 

QC2  12  of  12(111.4%) 

QC2  12  of  12  (89.4%) 

QC3  12  of  12  (-10.5%) 

QC3  12  of  12  (90.1%) 

QCl  11  of  12  (-52.1%) 

QCl  12  of  12  (29.2%) 

Precision  3 

QC2  12  of  12  (-30.7%) 

QC2  12  of  12  (37.6%) 

QC3  12  of  12  (-43.9%) 

QC3  12  of  12  (84.9%) 

QCl  12  of  12  (-43.9%) 

QCl  12  of  12  (30.6%) 

Precision  4 

QC2  12  of  12  (-42.7%) 

QC2  12  of  12  (14.8%) 

QC3  11  of  12  (-48.1%) 

QC3  11  of  12  (36.3%) 

Change  of 

QCl  N/A  due  fo  experimental  design 

QCl  N/A  due  fo  experimenfal  design 

inoculum 

QC2  N/A  due  to  experimental  design 

QC2  N/A  due  fo  experimenfal  design 

volume 

QC3  12  of  12  (-38.3%) 

QC3  12  of  12  (2.9%) 

Sfabilify 

QCl  12  of  12  (-54.3%) 

QCl  12  of  12  (109.9%) 
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QC2  12  of  12  (-50.8%) 

QC2  12  of  12  (83.0%) 

QC3  11  of  12  (-48.1%) 

QC3  11  of  12  (95.4%) 

Cell  Seeding 

Time  (24  h) 

QCl  12  of  12  (-17.3%) 

QCl  12  of  12  (94.3%) 

QC2  12  of  12  (-2.1%) 

QC2  12  of  12  (67.0%) 

QC3  12  of  12  (-12.0%) 

QC3  12  of  12  (68.1%) 

Cell  Seeding 

Time  (48  h) 

QCl  12  of  12  (9.0%) 

Nof  performed* 

QC2  12  of  12  (22.0%) 

QC3  12  of  12  (18.4%) 

Day  fo  Slain 

QCl  12  of  12  (-52.4%) 

QCl  12  of  12(141.6%) 

QC2  12  of  12  (-28.5%) 

QC2  12  of  12  (39.9%) 

QC3  12  of  12  (-40.0%) 

QC3  12  of  12  (29.2%) 

*The  MARV  validation  did  not  include  an  examination  to  determine  if  cells  plated  at  700,000  ± 
25,000  cells/well  could  be  used  successfully  af  48  hours  posf-seeding. 


The  largesf  observed  percenfage  difference  from  fhe  nominal  filer  was  seen  for  MARV  Precision  1 
QC3  af  175.5%,  equaling  fo  a  value  of  4.95  x  10^  PFU/mL.  This  filer  is  +0.151ogio  above  fhe  nominal 
tiler.  The  lowesf  observed  value  compared  fo  fhe  nominal  filer  was  fhaf  observed  for  EBOV 
Sfabilify  QCl,  which  was  -54.3%,  equaling  fo  a  value  of  5.35  x  10®  PFU/mL.  This  filer  is  -0.341ogio 
below  fhe  nominal  filer.  Based  on  fhese  real  fesfs,  fhe  accepfance  criferion  of  ±0.51ogio  of  fhe 
nominal  tiler  is  reasonable.  Over  time  and  through  use  of  fhe  assay,  it  may  be  possible  to  reduce 
the  acceptance  criterion  to  a  smaller  window,  but  at  this  time  the  criterion  of  ±0.51ogio  is  useful  fo 
describe  fhe  accuracy  limifs  of  fhe  assay. 


3.2  Confirmation  of  Precision 

For  fhe  precision  experiments,  QCl,  QC2,  and  QC3  were  analyzed  four  limes,  once  on  each  of  fwo 
different  days  by  two  analysts  per  day,  working  independently.  The  goal  of  fhese  experimenfs  was 
fo  evaluafe  fhe  performance  of  fhe  assay  on  differenl  days  when  fwo  independenf  assays  were  run 
in  parallel  by  fwo  separafe  analysis.  The  resulfs  of  fhese  four  tesfs  were  evaluafed  for  precision, 
defined  as  fhe  closeness  of  individual  fesf  resulfs  when  fhe  procedure  is  applied  repeafedly  fo 
multiple  aliquots  of  a  single  homogeneous  sample.  The  precision  of  a  mefhod  is  usually  expressed 
as  Sfd  Dev  or  %CV.  Precision  was  considered  for:  1)  repeafabilify  precision,  which  indicafes  how 
precise  fhe  fesf  resulfs  are  under  ideal  condifions  (same  sample,  analysf,  and  day);  and  2) 
infermediafe  precision,  which  expresses  wifhin-laborafory  variafions,  such  as  differenl  days, 
differenl  analysis,  differenl  insfrumenfs,  or  cells  from  differenl  passage  numbers.  The  accepfance 
criferion  for  Precision  in  fhis  validation  was  defined  as  a  measured  %CV  of  <  30%  for  each  QC  sef 
measured  in  fwo  paired  experimenfs  performed  by  fwo  analysis.  Figure  1  demonsfrafes  fhe 
similarifies  in  MARV  and  EBOV  QC  sef  filers  measured  by  analysis  A  and  B  across  four  fesfs.  The 
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CV%  measurements  are  indicated  directly  above  the  bars  in  the  graphs.  One  instance  of  %CV 
greater  than  30%  was  observed  for  the  EBOV  Test  4  Analyst  B  QCl  sample  (34.1%);  however,  this 
QC  sample  set  was  accepted  because  the  other  QC  sets  in  Test  4  passed,  and  other  analyst's  Test  3 
data  passed.  Moreover,  the  other  Precision  tests  and  the  Accuracy  tests  had  already  been 
performed  on  fhis  validation  and  the  %CV  values  all  passed  the  <  30%  criterion.  Subsequently,  out 
of  six  more  assays  performed  fhroughouf  fhe  EBOV  validafion,  if  was  found  fhaf  all  ofher  QC 
dafasets  passed  fhe  %CV  criterion. 


Figure  1.  The  MARV  and  EBOV  validated  plaque  assays  demonstrate  Repeatability  and 

Intermediate  Precision 


MARV  Precision  Data  EBOV  Precision  Data 
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Legend  for  Figure  1.  The  shaded  bars  represenf  average  measured  titers  for  QCl,  QC2  or  QC3  for 
MARV  or  EBQV,  measured  by  analysf  A  or  B  in  their  Test  performance  iteration.  Tests  1  and  2  were 
performed  on  fhe  same  day,  buf  independently  by  each  analyst.  Tests  3  and  4  were  similarly 
performed,  but  on  a  different  day.  The  three  horizontal  dotted  lines  plot  the  nominal  titers  of 
MARV  and  EBQV  QCl,  QC2  and  QC3.  Those  nominal  tilers  are  presenfed  in  Tables  1  and  2.  The 
%CVs  for  fhe  average  observed  QC  tilers  are  shown  above  fhe  relevant  bars,  and  the  values  were 
assigned  a  cutoff  for  accepfance  of  <  30%  for  this  cell-based  biological  assay. 

Analysis  of  dually-counted  data,  recorded  on  separate  data  sheets,  was  included  in  the  validation  to 
assess  the  consistency  of  plaque  counting  between  two  analysts.  Precision  of  plaque  counting 
performed  by  fwo  fechnicians  was  defermined  by  measuring  %  variability  of  mean  plaque  counts  in 
24  wells  when  counted  by  both  analysts  on  the  same  set  of  12  pairs  of  plafes  (two  duplicate  wells 
per  plate).  The  %  variability  was  calculated  as  follows: 


Variability  (%)  =  Mean  plaque#  Analyst  A-  Mean  plaque  #  Analyst  B  x  100 

((Mean  plaque  #  Analyst  A  +  Mean  plaque  #  Analyst  B)  ^  2) 

The  precision  test  was  designed  such  that  analysts  counted  plaques  from  their  own  experiments 
and  then  traded  plates  and  independently  counted  each  other's  plates.  The  sets  of  QCl,  QC2  and 
QC3  plaque  counfs  recorded  by  each  analysf  (n=24  per  QC  set/analyst)  were  averaged,  and  fhe  dafa 
were  analyzed  for  %  variability  between  analysts  of  fhe  overall  plaque  numbers  per  QC  sample  per 
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test.  The  absolute  %  variability  was  required  to  be  less  than  30%  to  pass  this  acceptance  criterion. 
The  data  demonstrated  there  was  no  significant  effect  on  the  tallied  plaque  numbers  when  the 
EBOV  and  MARV  plaques  were  counted  by  different  analysts  (Table  4). 


Table  4.  Variability  in  counts  measured  by  two  analysts  is  minimal  for  fhe  EBOV  and  MARV 

plaque  assays. 


%  Variability  in  EBOV  plaque 
counts  between  two  analysts 

%  Variability  in  MARV  plaque  counts 
between  two  analysts 

Test  Counted  by  both 
Analysts 

QCl 

QC2 

QC3 

QCl 

QC2 

QC3 

Test  1  (run  by  Analyst  A) 

0.6% 

2.9% 

9.3% 

17.8% 

15.7% 

13.5% 

Test  2  (run  by  Analyst  B) 

6.0% 

3.4% 

1.9% 

28.2% 

27.4% 

26.8% 

Test  3  (run  by  Analyst  A) 

6.5% 

7.2% 

4.2% 

9.5% 

7.3% 

4.8% 

Test  4  (run  by  Analyst  B) 

3.6% 

6.5% 

1.3% 

15.1% 

8.3% 

8.6% 

The  Bland- Alfman  plof  (Figure  2)  below  is  a  visual  represenfafion  of  fhe  agreemenf  of  fhe  EBOV 
plaque  counfs  made  by  fhe  fwo  analysfs  for  each  Precision  experimenf.  The  difference  befween 
counfs  of  each  well  are  plotted  on  fhe  y-axis,  while  fhe  mean  of  fhe  fwo  counfs  for  each  well  are 
plotted  on  fhe  x-axis.  The  cenfral  reference  line  in  each  plof  is  fhe  mean  of  fhe  differences  befween 
counfs;  while  fhe  fwo  ofher  reference  lines  represenf  fwo  sfandard  deviations  of  fhe  differences 
above  and  below  fhe  mean.  If  appears  fhaf  fhe  fewer  fhe  plaques  counfed,  fhe  closer  fhe  dafa  fell  fo 
fhaf  cenfral  reference  line:  fherefore,  if  is  possible  fhaf  when  fhere  are  fewer  plaques  fo  counf,  fhere 
is  less  variation  befween  fwo  analysts.  The  Bland- Altman  analysis  was  also  completed  for  fhe 
MARV  dafa  set  (see  Supplementary  File  1),  and  the  results  were  very  similar. 
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Figure  2.  Bland- Altman  analysis  demonstrates  a  visual  representation  of  the  agreement  between 
plaque  numbers  counted  by  two  individual  analysts. 


Mean  of  Counts 


Sample  •  QC1  •  QC2  •  QC3 


In  addition,  an  intra-class  correlation  (ICC)  analysis  was  performed  for  evaluating  the  Precision  of 
fhe  assays  for  EBOV  and  MARV  (Table  5).  The  ICC  fechnique  operafes  on  dafa  organized  into  test 
groups.  It  assesses  the  consistency  or  conformity  of  measurements  made  by  multiple  observers 
measuring  the  same  quantity  [12].  ICC  was  used  to  analyze  the  results  when  variation  (e.g. 
multiple  analysts)  was  introduced  to  a  set  for  the  same  samples.  Evaluation  of  the  calculated  ICC 
values  passed  the  test-specific  accepfance  criterion  of  af  leasf  0.8  for  the  lower  95%  Cl,  indicating 
that  plaque  count  data  from  two  different  analysts  for  the  same  sets  of  plafes  are  in  high  agreemenf 
and  fhere  is  very  litfle  variafion  befween  analysis  when  plaques  are  counfed.  Inherently,  there 
must  be  clear  plaques  of  high  quality  that  are  recognizable  by  a  trained  analyst.  When  trained 
individuals  were  presented  with  high  quality  plaques,  both  analysts  generated  similar  plaque 
counts  and  calculated  titer  data.  Throughout  the  performance  of  fhe  validafed  assay,  if  will  be 
possible  fo  have  mulfiple  users  perform  fhe  assay  for  various  dafa  collecfion  needs  and  fhe  dafa  will 
be  comparable  because  the  assay  has  repeatability  and  intermediate  precision.  ICC  coefficients  and 
95%  confidence  intervals  (Cl)  were  calculated;  demonstrating  how  strongly  the  EBOV  and  MARV 
plaque  count  results  resemble  each  other  when  counted  by  two  analysts  in  the  Precision  tests  (Table 
5). 


10 


UNCLASSIFIED 


DISTRIBUTION  STATEMENT  A:  Approved  for  public  release;  distribution  is  unlimited. 


Journal  2015,  volume,  page-page 


Table  5.  ICC  analysis  reveals  a  high  level  of  agreement  in  plaque  counts  between  two  analysts 

collecting  data  using  the  same  methods. 


EBOV 

95%  Confidence 

Limits 

MARV 

95%  Confidence 

Limits 

Precision 

Test/Analyst 

Intra-class 

Correlation 

Lower 

Upper 

Precision 

Test/Analyst 

Intra-class 

Correlation 

Lower 

Upper 

Test  1 

0.99013 

0.98809 

0.99182 

Test  1 

0.99013 

0.98809 

0.99182 

Test  2 

0.99490 

0.99384 

0.99578 

Test  2 

0.99490 

0.99384 

0.99578 

Test  3 

0.96788 

0.96130 

0.97334 

Test  3 

0.96788 

0.96130 

0.97334 

Test  4 

0.96507 

0.95796 

0.97100 

Test  4 

0.96507 

0.95796 

0.97100 

3.3  Confirmation  of  Linearity 

The  measured  average  titers  of  EBOV  and  MARV  QCl,  QC2,  and  QC3  (n=12,  from  the  Accuracy 
data  set.  Tables  1  and  2)  were  proportional  to  their  respective  nominal  high,  medium,  and  low 
titers,  which  demonstrated  the  linearity  of  the  assay.  For  the  EBOV  validation  studies,  the  slope  and 
R2  values  were  calculated  at  0.52  and  0.8275,  respectively.  The  R^  value  had  to  be  >  0.5  to  pass  the 
acceptance  criterion  for  linearity.  Similarly  for  MARV,  the  slope  and  R^  values  were  calculated  at 
0.81  and  0.8229,  respectively,  supporting  that  both  validated  plaque  assays  demonstrated  linearity 
in  sample  measurement. 

3.4  Robustness  in  the  Assay 

3.4.1.  Stability.  Stability  of  the  virus  stock  after  multiple  freeze-thaw  cycles  was  tested  to 
determine  if  the  virus  samples  and  positive  controls  were  stable  during  common  manipulations. 
Repeat  measurement  of  samples  is  sometimes  required  during  the  data  collection  process, 
especially  if  an  assay  fails  or  other  factors  beyond  a  researcher's  control  necessitate  a  repeat 
performance.  For  the  EBOV  validation,  only  virus  diluted  in  cell  culture  medium  was  tested  for 
freeze-thaw  cycle  stability  (Figure  3).  For  MARV  validation  studies,  cell  culture  medium,  NHP 
serum  and  NHP  plasma  were  all  matrices  tested  for  effect  on  virus  stability  after  three  freeze-thaw 
cycles  (Figure  4).  These  matrices  were  spiked  with  virus  to  a  titer  consistent  with  QC2.  For  the 
EBOV  studies,  a  very  similar  titer  was  measured  for  samples  which  had  been  continuously  held 
frozen,  and  those  which  had  been  through  three  cycles  of  freeze-thawing.  The  thawing  procedure 
was  for  no  less  than  30  minutes  at  ambient  temperature,  which  was  adequate  to  ensure  complete 
thawing  of  the  sample,  and  the  refreezing  was  for  no  less  than  20  hours  at  -60°C  or  lower,  on  three 
consecutive  days  (denoted  as  F/S  in  Figure  4). 

The  validation  protocol  acceptance  criterion  for  Stability  was  that  the  F/S  sample  had  to  return  a 
titer  no  more  than  30%  different  from  the  value  measured  for  the  non-freeze-thawed  sample  to 
demonstrate  that  freeze-thawing  had  no  detrimental  effect  on  the  sample.  Consistently,  less  than  a 
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30%  difference  was  observed  for  fhe  EBOV  and  MARV  cell  culfure  samples,  where  fhe  widesf  range 
of  differences  from  the  unfreated  samples  was  15.4%,  and  12.4%,  for  EBOV  QC2  and  fhe  MARV 
QCl,  respecfively  (MARV  dafa  shown  in  Supplemenfary  File  2).  Sfafisfical  comparison  by  Sfudent's 
T-tesf  of  fhe  freafed  versus  untreafed  samples  demonsfrafed  sfafisfical  significance  befween  fhe  two 
datasets  (p=0.04  for  QCl  and  p<0.01  for  QC2),  even  fhough  the  titers  measured  for  the  two  groups 
were  very  close,  thus  indicating  very  low  variability  in  titers  measured  for  each  sample  group  (see 
also  error  bars  in  Figure  3).  The  sfafisfical  significance  in  fhis  case  did  nof  present  a  meaningful 
biological  difference  in  the  titers  of  fhe  treafed  samples.  The  pracfice  of  using  a  cell  culfure  sample 
thaf  has  up  fo  fhree  freeze-fhaw  manipulafions  is  acceptable  for  fhe  EBOV  and  MARV  validafed 
assays. 


Figure  3.  Freeze-fhaw  heated  and  untreated  QC  samples  have  very  similar  measured  titers  in  the 
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The  stability  of  MARV-spiked  cynomolgus  macaque  or  rhesus  macaque  serum  or  plasma  samples 
(CMST/CMPT)  or  (RMST/RMPT)  was  tested  using  virus  at  a  mid-level  concentration  (the 
concentration  of  QC2).  To  pass  the  acceptance  criterion  for  this  comparison  assay  in  the  validation, 
the  titers  measured  for  fhe  freeze-fhawed  samples  had  fo  fall  within  30%  of  fhe  values  measured  for 
the  untreated  samples.  For  CMST  and  CMSTF/S  the  average  titers  were  5.03  x  10^  PFU/mL 
(CV=10.9%)  and  4.44  x  10^  PFU/mL  (CV=21.3%)  respectively,  comprising  only  an  11.7%  difference 
befween  the  values  for  the  two  conditions  (Supplementary  Figure  3).  For  CMPT  and  CMPTF/S  the 
average  titers  were  9.57  x  10^  PFU/mL  (CV=7.3%)  and  1.01  x  10^  PFU/mL  (CV=12.2%)  respectively, 
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with  a  5.8%  difference  between  the  values  for  the  two  conditions  (Supplementary  Figure  3).  For 
RMST  and  RMSTF/S  the  average  titers  were  1.01  x  10*’  PFU/mL  (CV=13.2%)  and  8.63  x  10®  PFU/mL 
(CV=9.3%)  respectively,  comprising  only  a  14.6%  difference  (Figure  4).  For  RMPT  and  RMPTF/S  the 
average  titers  were  1.48  x  10®  PFU/mL  (CV=19.0%)  and  1.40  x  10®  PFU/mL  (CV=14.2%)  respectively, 
with  a  5.4%  difference  befween  the  values  (Figure  4).  No  statistically  significant  difference  (where  p 
<  0.05)  befween  tifers  was  observed  for  the  cynomolgus  macaque  samples,  or  the  rhesus  macaque 
plasma  samples.  A  statistically  significant  difference  was  observed  between  titers  measured  from 
the  RMST  and  RMSTF/S  sample  set,  where  the  p-value  was  calculated  at  <0.01.  Based  on  the  assay 
performance,  the  measured  titers  of  RMST  and  RMSTF/S  were  nof  biologically  different.  They  were 
within  30%  of  each  other,  even  though  mathematically  they  are  distinct  data  sets  with  low  variation 
in  the  numbers.  In  total,  these  results  indicated  that  it  is  not  a  detrimental  practice  to  incur  three 
freeze/thaw  cycles  on  cell  culture  medium,  or  nonhuman  primate  serum  and  plasma  samples 
containing  a  filovirus  concentration  of  abouf  10®  fo  10®  PFU/mL  during  regular  use. 


Figure  4.  MARV-spiked  rhesus  macaque  serum  and  plasma  can  be  fifered  affer  freeze-fhaw  cycles. 
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Figure  4.  Legend.  Rhesus  macaque  serum  (RMST)  and  plasma  (RMPT)  were  subjected  to  three 
freeze-thaw  cycles  of  being  held  fhawed  af  ambient  temperature  for  af  leasf  30  minufes,  and 
refrozen  af  -60°C  or  colder  for  no  less  than  20  hours,  on  three  consecutive  days.  Samples  were 
titered  in  the  plaque  assay  after  the  fourth  thaw.  There  was  little  variability  in  each  average  titer, 
and  a  Student's  T-test  found  a  statistically  significant  difference  (p  <  0.01)  between  RMST  and 
RMSTF/S  sample  sets,  yet  the  titers  are  acceptably  similar;  therefore,  fhe  difference  is  nof 

biologically  significant. 

3.4.2.  Change  of  inoculum  volume  and  tissue  culture  vessel.  It  is  sometimes  necessary  to  quantitate 
virus  solutions  with  potentially  very  low  titers  using  the  plaque  assay.  Increasing  the  inoculum 
volume  in  the  plaque  assay  to  200  pi  or  1  mL  lowers  the  lower  limit  of  quantifafion  (LLOQ)  to  50 
PFU/mL  or  10  PFU/mL,  respectively,  when  plating  undiluted  samples  and  detecting  a  minimum  of 
10  plaques  in  the  vessel.  Flowever,  larger  volumes  need  to  be  plated  onto  a  sufficiently  larger 
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surface  area  of  cells  to  maintain  the  right  plating  conditions  for  plaque  formation.  We  previously 
evaluated  the  performance  of  various  inoculum  volumes  and  determined  that  larger  volumes  (e.g.  > 
400  uL)  plated  onto  a  small  area,  e.g.  a  6-well  plate  with  a  surface  area  of  only  9.5  cmVwell,  may  not 
promote  adequate  surface  contact  of  virus  particles  to  force  them  to  be  directed  onto  cell 
monolayers  to  make  the  optimal  number  of  plaques  [8].  The  effect  of  inoculum  volume  variation 
was  tested  in  EBOV  and  MARV  validation  studies  by  plating  serial  tenfold  dilutions  of  QC3  in  100 
pL,  200  pL  inoculum  volumes  on  cells  in  6-well  plates  (growth  area  of  9.5  cmVwell)  or  1  mL 
volumes  on  100mm  dishes  (55  cmVwell).  For  this  experiment,  one  dilution  series  of  QC3  was 
prepared  in  a  volume  sufficient  to  inoculate  all  vessels  with  the  same  dilution  preparation  at  the 
same  time.  The  test-specific  acceptance  criterion  for  this  validation  test  was  that  80%  of  the 
observed  titers  in  each  of  the  100  pi,  200  pi  and  1  mL  sets  must  be  within  ±  0.5  logio  of  the  nominal 
titer  for  QC3  to  demonstrate  the  accuracy  of  the  method  using  the  larger  inoculum  volume.  All  of 
these  QC3  data  points  passed  this  criterion  (Figure  5),  and  therefore  the  use  of  200  pi  or  1  mL  as  the 
inoculum  volume/vessel  combination  was  an  acceptable  practice  resulting  in  accurate  viral  titration 
for  EBOV  and  MARV,  as  compared  to  the  other  data  collected  in  this  validation  where  100  pi  in  a  6- 
well  plate  was  used  most  often.  The  observed  values  for  the  three  conditions  in  the  MARV 
validation  were  all  very  similar,  but  the  observed  values  were  slightly  different  for  the  three 
conditions  in  the  EBOV  validation. 


Figure  5.  Various  inoculum  volumes  and  tissue  culture  vessels  are  suitable  for  use  in  the  validated 

assay. 
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Figure  5  legend.  The  individual  titers,  averages  and  standard  deviations  for  n=up  to  15  6-well  plates 
or  100mm  dishes  are  presented  for  inoculum  volumes  of  100  pi  (circles),  200  pi  (squares)  and  ImL 
(triangles)  for  MARV  and  EBOV  validation  data.  The  central  dotted  line  in  the  graphs  represents 
the  nominal  titer  for  QC3  for  MARV  and  EBOV,  respectively.  The  dotted  lines  above  and  below  the 
central  line  represent  ±  0.5  logio  for  the  nominal  QC3  titer,  and  the  boundaries  into  which  80%  of  the 
titers  must  fall  to  pass  the  accuracy  criterion  in  this  assay.  The  QC3  titers  the  dotted  lines  represent 

are  presented  in  Tables  1  and  2  for  reference. 
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Evaluation  of  different  volumes  and  larger  culture  dishes  enabled  examination  of  the  LLOQ  and 
the  limit  of  detection  (LOD)  of  filovirus  plaques  in  this  assay  system.  The  LLOQ  in  this  experiment 
depended  on  the  vessel  used,  the  inoculum  volume  plated  and  the  observation  of  at  least  10 
plaques,  as  required  by  the  plaque  assay  SOP.  Using  all  of  the  raw  plaque  count  data  on  the 
validation  studies,  it  was  observed  that  as  few  as  1,  2  or  3  plaques  could  be  detected  in  wells  or 
dishes  at  the  lowest  dilutions  tested.  It  is  important  that  an  investigator  be  able  to  distinguish  the 
presence  of  only  one  plaque  compared  to  a  monolayer  blemish  or  other  area  of  damage.  The 
negative  control  wells  and  dishes  in  these  experiments  were  free  of  holes,  blemishes  or  damage  that 
could  be  confused  for  a  singular  plaque.  The  presence  of  singular  or  a  small  number  of  plaques  at 
the  lowest  dilutions  was  believable  due  to  the  presence  of  increasingly  higher  numbers  of  plaques 
in  the  more  concentrated  dilutions  in  the  series.  For  example,  in  dishes  the  average  EBOV  plaque 
number  in  1  mL  of  inoculum  was  120  plaques  at  the  10'^  dilution,  12.3  at  the  lO'^  dilution,  and  2.92  at 
the  10"^  dilution.  The  2.92  result  was  averaged  from  a  series  of  13  dishes  with  plaque  numbers 
ranging  from  five  to  only  one  plaque  per  dish.  The  presence  of  so  few  true  plaques  was  credible, 
and  supports  an  LOD  of  as  little  as  one  plaque,  because  it  was  in  the  context  of  more  plaques  seen  at 
each  of  the  next  higher  dilutions.  However,  using  counts  this  low  is  not  recommended  for 
calculation  of  an  actual  titer.  The  SOP  supports  counting  plaques  greater  than  10  for  assurance  that 
a  statistically  significant  number  of  plaque  events  results  in  a  robust  quantifiable  data  result.  For  the 
LLOQ,  10  plaques  in  a  0.2  mL  volume  of  undiluted  sample,  plated  in  a  6-well  plate  results  in  a 
LLOQ  of  50  PFU/mL,  whereas  plating  1  mL  of  undiluted  sample  in  a  100mm  culture  dish  results  in 
a  LLOQ  of  10  PFU/mL.  The  LOD  of  merely  1  PFU/mL  in  the  100mm  dish  is  believable  in  the  context 
of  a  dilution  series.  This  validation  supports  a  LOD  of  one  plaque  per  unit  volume  plated,  but 
investigators  should  strive  to  plate  a  dilution  series  to  assure  the  presence  of  as  many  plaques  as 
possible,  especially  when  titrating  a  sample  of  unknown  virus  content. 

3.4.3.  Optimal  Day  to  stain  and  count  plaques. 

Building  flexibility  of  staining  and  counting  times  into  the  validated  assay  is  desirable  for  ease  of 
performance,  provided  that  these  practices  do  not  incur  unwanted  variability.  We  previously 
reported  that  using  a  final  concentration  of  0.5%  agarose,  and  4  to  5%  (vol/vol)  Gibco  or  Sigma 
neutral  red  vital  dye  in  the  secondary  overlay  produces  optimal  staining  conditions  [8].  In  addition, 
we  observed  that  the  most  optimal  days  to  stain  were  Day  7  and  8  post-inoculation,  and  reading 
within  two  days  of  staining  [8].  These  day-to-stain  robustness  tests  were  performed  to  determine  if 
there  was  any  difference  in  observed  titers  collected  when  plaques  were  stained  on  Day  7,  and 
counted  on  Days  8  or  9,  versus  stained  on  Day  8  and  counted  on  Days  9  or  10.  Two  sets  of  75  plates 
each  of  Vero  E6  cell  plates  were  produced  to  perform  both  of  these  tests  on  the  same  day  so  that 
neither  cell  passage  number  nor  day  of  performance  could  be  considered  variables,  thus  all  150 
plates  were  inoculated  with  the  same  virus  preparations  of  PC,  QCl,  QC2  and  QC3.  One  set  of 
plates  was  stained  on  Day  7,  for  counting  on  Days  8  and  9,  and  one  set  was  stained  on  Day  8,  for 
counting  on  Days  9  and  10.  The  effect  of  altering  staining  day  on  calculated  titer  was  evaluated 
through  a  comparison  of  titers  at  each  of  the  days  counted.  The  test-specific  acceptance  criterion  for 
this  assay  stated  if  counting  on  the  latter  day  produced  results  greater  than  ±  20%  different  than  the 

15 


UNCLASSIFIED 


DISTRIBUTION  STATEMENT  A:  Approved  for  public  release;  distribution  is  unlimited. 


Journal  2015,  volume,  page-page 

earlier  day,  then  counting  on  the  earlier  day  will  be  chosen  as  the  validated  method.  Percent 
variability  was  calculated  as  follows: 

Variability  (%)  =  (Mean  titer  Day  8  -  Mean  titer  Day  91  x  100 
((Mean  titer  Day  8  +  Mean  titer  Day  9)/2) 


Generally,  many  filovirus  researchers  with  experience  using  the  plaque  assay  accept  that  MARV 
and  EBOV  plaque  assay  plates  should  be  stained  on  Day  7  or  Day  8,  and  the  plaques  can  be 
therefore  counfed  on  Day  8  or  Day  9,  respecfively.  The  resulfs  from  fhis  well-confrolled  study 
generally  agree  with  this  practice.  For  most  of  fhe  EBOV  and  MARV  QC  sample  sefs,  fhere  was  less 
fhan  20%  variabilify  in  fhe  resulfs  when  fhe  same  plafes  sfained  on  Day  7  were  counfed  on  Day  8  or 
again  on  Day  9.  The  same  was  frue  for  plafes  sfained  on  Day  8  and  counfed  on  Day  9  or  again  on 
Day  10,  buf  fhe  Day  10  dafa  collecfion  fime  had  limifafions.  Some  EBOV  and  MARV  QC  dafasefs 
failed  af  fhe  Day  10  collecfion  fime  due  fo  an  increase  in  unreadable  wells  on  fhe  plafes.  More  wells 
on  Day  10  were  uncounfable  (U)  due  fo  fofal  monolayer  damage.  These  were  fhe  same  monolayers 
which  had  looked  good  and  were  counfable  on  Day  9  (Table  6).  In  general,  fhe  monolayers  wifh 
sfain  on  fhem  for  abouf  48  hours  looked  worse  and  had  more  incidenfs  of  U  scores  fhan  fhose  which 
had  been  under  sfain  for  only  abouf  24  hours.  Day  10  monolayers  nof  only  had  been  under  sfain  for 
48  hours,  buf  also  were  an  addifional  day  old,  increasing  fhe  possibility  of  U  monolayers.  This 
phenomenon  of  an  increase  in  U  scores  in  all  of  fhe  144  wells  on  fhe  second  day  was  worse  for 
MARV  fhan  EBOV,  based  on  fhe  dafa  in  Table  6,  and  especially  for  MARV  QCl.  If  is  difficulf  fo 
sfafe  fhe  reason  for  fhis  observafion,  buf  if  is  possible  fhaf  MARV  plaques  grow  or  spread  fasfer  and 
ruin  the  monolayer  to  the  extent  that  it  is  uncountable  by  the  second  day  under  stain.  Waiting  until 
Day  10  to  count  plates  can  lead  to  data  loss  and  would  be  a  risky  practice,  and  therefore  is  nof 
recommended  in  fhe  validafed  assay  mefhod. 


Table  6.  Day  10  monolayers  become  fragile  and  deferiorafe,  increasing  fhe  number  of  uncounfable 


(U)  wells  across  all  dilufions  plafed  in  fhe  assay. 


EBOV  1 

#  U  wells  in  144  wells  plated 
(%  of  total  wells) 

QCl 

QC2 

QC3 

Stained  Day  7,  counted  Day  8 

5  (3.5%) 

0  (0%) 

1  (0.7%) 

Stained  Day  7,  counted  Day  9 

4  (2.8%) 

0  (0%) 

2  (1.4%) 

Stained  Day  8,  counted  Day  9 
Stained  Day  8,  counted  Day 

10 

6  (4.2%) 

66  (45.8%) 

0  (0%) 

79  (54.9%)* 

2  (1.4%) 

56  (38.9%)* 

MARV 

#  U  wells  in  144  wells  plated 
(%  of  total  wells) 

QCl 

QC2 

QC3 

Stained  Day  7,  counted  Day  8 

24  (16.7%) 

1  (0.7%) 

4  (2.8%) 

Stained  Day  7,  counted  Day  9 

67  (46.5%) 

5  (3.5%) 

4  (2.8%) 

Stained  Day  8,  counted  Day  9 
Stained  Day  8,  counted  Day 

58  (40.3%) 

96  (66.7%) 

1  (0.7%) 

56  (38.9%) 

23  (16.0%) 

41  (28.5%)* 

10 
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^Uncountable  (U)  wells  in  the  best  countable  dilutions  on  the  plates  were  so  numerous  that  the 
indicated  QC  set  failed  to  provide  the  minimum  number  of  dafa  poinfs  to  satisfy  fhe  acceptance 
criteria  for  successful  day-fo-sfain  assay  performance. 

The  majorify  of  the  comparisons  of  tifers  counfed  af  Day  8,  9  or  10  of  the  assay  were  within  20%  of 
fhe  firsf  counfed  value  (Figure  6  and  Table  7).  There  was  one  insfance  in  fhe  EBOV  dafa  where  fhe 
variability  was  calculated  to  be  22%  higher  on  Day  9  than  it  was  when  counted  on  Day  8.  However, 
this  phenomenon  of  >  20%  increase  was  nof  observed  again  with  the  QC2  and  QC3  sets  stained  on 
Day  7  and  counted  on  Day  8  versus  again  on  Day  9.  These  %  increases  were  16.8%  and  19.9%, 
respectively.  If  the  practice  of  waiting  one  extra  day  to  count  the  plaques  were  truly  introducing 
variability  in  titer  over  the  original  value  collected  at  the  first  day  after  staining,  this  observation 
would  have  been  more  robust  and  it  would  have  been  noted  for  all  three  of  the  QC  sets,  no  matter 
what  their  nominal  titer.  There  were  no  instances  in  the  MARV  dataset  where  the  titer  was  greater 
than  20%  when  counted  on  the  latter  day.  Moreover,  the  Test  results  for  all  condifions  fell  wifhin 
fhe  Accuracy  criferia  for  each  QC  sef,  meaning  thaf  fhe  tifers  measured  for  each  QC  sef  were  wifhin 
±  0.5  log  of  fhe  nominal  filer  for  each  QC  sef.  The  assay  is  valid  for  plafes  sfained  on  Day  7  and 
counfed  on  Days  8  and  9,  or  stained  on  Day  8  and  counted  on  Day  9. 

Figure  6.  QC  sample  titers  were  acceptably  similar  if  counfed  one  or  two  days  post-staining. 


I  QCl  I  I  QC2  I  I  QC3  I 


PRn\/ 


Figure  6.  The  averages  and  standard  deviations  (n=12)  for  QCl-3  titers 
stained  and  counted  on  Days  8,  9  or  10  are  presented.  The  three 
horizontal  dotted  lines  plot  the  nominal  titers  of  MARV  and  EBOV 
QCl,  QC2  and  QC3  (Tables  1  and  2).  *  indicates  failed  dataset,  as 
described  in  Table  7. 


stained  D7.  counted  Day  8 


□  Stained  D7.  counted  Day  9 
|__|  Stained  D8.  counted  Day  9 
Istalned  D8.  counted  Day  10j 


Table  7.  Counting  on  one  and  two  days  following  sfaining  is  an  acceptable  practice  for  fhe 


validafed  plaque  assay. 


First  plate  set 

Second 

?late  set 

Counted  Dav  8 

(stained  Day  7) 

Counted  Dav  9 

(stained  Day  7) 

Counted  Dav  9 

(stained  Day  8) 

Counted  Dav  10 

(stained  Day  8) 

EBOV 

QC1=  5.57x105 

QC1=  6.95  X  105 

QC1=  5.63  X  105 

QC1=  6.71  X  105 
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22%  titer  increase 

17.5%  titer  increase 

QC2=  7.65  X  104 

QC2=  9.05  X  104 

16.8%  titer  increase 

QC2=  8.13  X  104 

QC2=  dataset  failed* 

QC3=  6.25  X  103 

QC3=  7.63  X  103 

19.9%  titer  increase 

QC3=  8.74  X  103 

QC3=  dataset  failed* 

MARV 

QC1=  8.46  X  lOf- 

QC1=  9.79  X  105 

14.5%  titer  increase 

QC1=  8.95  X  105 

QC1=  1.02  X  103 

13.2%  titer  increase 

QC2=  4.90  X  105 

QC2=  5.50  X  105 

11.5%  titer  increase 

QC2=  6.06  X  105 

QC2=  7.22  X  105 

17.5%  titer  increase 

QC3=  4.52  X  104 

QC3=  5.02  X  104 

10.4%  titer  increase 

QC3=  4.40  X  104 

QC3=  dataset  failed* 

^Datasets  failed  due  to  presence  of  too  many  unscorable  wells,  and  too  few  counts  resulting  from 
the  assay  set  to  meet  assay  performance  acceptance  criteria. 


3.4.4.  Cell  seeding  and  time  of  cell  use. 

Validation  tests  were  performed  to  evaluate  the  performance  of  cells  at  a  particular 
concentration  and  passage  number  when  used  at  24,  48  or  72  hours  after  seeding.  These 
experiments  were  done  to  test  assay  flexibility.  Building  in  the  option  to  use  cells  one  day  after  the 
planned  usage  could  ease  research  performance  in  unexpected  circumstances  where  the  need  for 
the  plaque  assay  performance  becomes  delayed  by  one  day  (e.g.  delay  due  to  inclement  weather  or 
illness).  Plated  cells  represent  time,  effort  and  money,  which  get  wasted  if  they  cannot  be  used,  and 
replating  could  delay  a  study  by  a  week  or  more. 

Tests  on  the  EBOV  validation  were  performed  to  evaluate  the  performance  of  cells  at  a  particular 
concentration  and  passage  number  (e.g.  passage  33)  used  24  hours  ±  6  hours  after  seeding,  or  used 
48  hours  ±  6  hours  after  seeding.  The  concentration  of  cells  plated  was  700,000  ±  25,000  cells/well. 
The  MARV  validation  experiment,  which  was  completed  before  the  EBOV  validation  was 
undertaken,  differed  slightly  in  that  the  cell  plating  ages  tested  were  24  and  72  hours  ±  6  hours  after 
seeding,  but  the  48  hour  range  was  not  tested.  For  both  validations,  the  first  batch  of  cells  was 
observed  at  24  hours  post-seeding  to  be  96-98%  confluent,  and  was  used  at  that  time.  The  second 
batch  was  observed  at  48  (or  72  for  MARV)  hours  post-seeding,  and  these  were  found  to  be  heavily 
confluent  (100%)  and  to  appear  tightly  packed  when  they  were  used,  especially  for  the  72  hour  old 
cells.  The  MARV  and  EBOV  assays  evaluated  the  titers  of  the  PC,  and  QC1-QC3  on  each  of  these 
cell  ages.  The  results  for  the  24  hour  and  48  hour  old  cells  tested  on  the  EBOV  validation  passed  all 
assay  performance  acceptance  criteria  (Figure  7).  The  titers  for  EBOV  PC  and  QC1-QC3  were  all 
acceptably  within  the  nominal  titer  for  each  QC  set,  whether  they  were  measured  on  24  hour  old 
cells  or  48  hour  old  cells.  The  titers  for  MARV  PC  and  QC1-QC3  measured  on  24  hour  old  cells 
were  all  acceptably  within  the  nominal  titer  for  each  QC  set,  but  the  72  hour  old  cells  did  not 
produce  passing  results  for  the  PC  or  any  of  QCl-3  because  all  the  titers  were  too  high  to  fall  within 
±0.51og  10  of  the  nominal  titers  of  those  samples. 
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It  is  possible  that  performance  of  fhe  MARV  assay  on  48  hour  old  cells  in  a  validafion  bridging 
experimenf  would  pass,  and  fhis  is  a  fufure  direcfion  which  will  be  evaluafed  fo  expand  fhe  MARV 
validation  package.  Another  concept  that  should  be  tested  in  future  validation  bridging 
experiments  for  MARV  and  EBOV  would  be  fo  plafe  cells  af  lower  concenfrafions  (e.g.  300,000 
cells/well),  evaluafe  by  microscopic  observafion  whefher  fhey  can  achieve  adequafe  confluency  by 
72  hours,  and  defermine  if  passing  fifer  resulfs  can  be  obfained.  Successful  resulfs  in  an  experimenf 
designed  in  fhis  manner  would  help  build  flexibility  into  the  performance  schedule  for  assays  on  a 
weekly  cadence.  Currently,  assays  must  be  performed  based  on  fhe  cell  seeding  schedule  in  fhe 
USAMRIID  cell  culfure  lab,  limifing  fhe  days  of  fhe  week  on  which  CLP  complianf  assays  may  be 
performed. 


Figure  7.  Use  of  cells  af  24  or  48  hours  posf  seeding  provides  accurafe  fifer  dafa  for  fhe  EBOV 
validafed  plaque  assay,  buf  use  af  72  hours  fails  in  fhe  MARV  assay. 


MARV 


EBOV 


Legend  for  Figure  7.  The  bars  represenf  average  measured  fifers  for  QCl,  QC2  or  QC3  for  MARV  or 
EBOV,  measured  on  cells  plafed  af  24,  48  or  72  hours  prior  fo  infecfion.  The  fhree  horizonfal  doffed 
lines  plof  fhe  nominal  fifers  of  MARV  and  EBOV  QCl,  QC2  and  QC3.  Those  nominal  fifers  are 
presenfed  in  Tables  1  and  2.  The  *  indicafes  failed  runs  for  MARV  fesfed  on  cells  plafed  72  hours 

before  use. 


3.4.5.  Cell  passage  number. 

As  presenfed  in  a  previous  filovirus  plaque  assay  sfandardizafion  reporf  [8],  and  promofed  as  an 
ideal  sfandardized  reagenf  [13],  fhe  BEI  Vero  E6  cells  are  fhe  Filovirus  Animal  Nonclincal  Group's 
(FANG)  cell  line  of  choice  for  performance  of  fhe  sfandardized  filovirus  plaque  assay.  Thus,  fhis  cell 
line  was  chosen  for  fhe  validafion  activifies  for  MARV  and  EBOV.  Tesfs  on  fhe  MARV  validafion 
were  performed  wifh  Vero  E6  cells  from  BEI  (G1008)  cafalog  no.  NR-596,  buf  fhey  were  lof  number 
3956812,  which  has  nof  been  as  well  characferized  as  fhe  now-preferred  lof  number  3956593.  The 
lof  3956812  cell  passage  ages  fesfed  in  fhe  MARV  validafion  experimenfs  ranged  from  21  fo  49,  and 
fhose  resulfs  agree  closely  wifh  published  dafa,  indicafing  fhaf  any  passage  age  in  fhaf  range 
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produced  passing  titers  for  the  MARV  PC  and  QCl-3  [8].  A  bridging  MARV  validation  experiment 
with  the  preferred  lof  of  cells  was  complefed  and  indicafed  fhaf  fhe  new  lof,  grown  under  GLP- 
complianf  condifions  af  USAMRIID,  funcfioned  similarly  af  fhe  passage  age  of  31,  wifh  older 
passages  fo  be  fesfed  in  fhe  fufure.  The  EBOV  validafion  experimenfs  began  using  Vero  E6  cells 
from  BEI  lof  3956812,  and  fesfed  passages  32-36. 

Tracking  fhe  performance  of  fhe  PC  on  a  variable  such  as  cell  passage  number  comprises  a 
sysfems  suifabilify  evaluafion  for  fhe  validafion.  Sysfem  suifabilify  helps  fhe  user  evaluafe  fhe 
performance  of  fhe  assay  over  fime,  and  if  a  factor  like  fhe  PC  fifer  begins  fo  fail  roufinely,  fhen  if 
can  be  an  indicafor  fhaf  somefhing  in  fhe  sysfem  (e.g.  fhe  cell  passage  number,  or  a  similar  assay 
consfifuenf)  is  no  longer  providing  a  consisfenf  assay  sysfem.  The  value  for  fhe  EBOV  PC  in  fhis 
validafion  had  fo  fall  wifhin  ±  0.5  logic  of  fhe  nominal  tifer  of  fhe  PC,  or  fhe  performance  of  fhe 
assay  was  deemed  as  a  failure.  The  range  in  which  fhe  PC  fifer  was  required  fo  fall  was  34,469  fo 
344,688  PFU/mL,  wifh  a  nominal  fargef  fifer  of  1.09  x  10®.  The  PC  fifer  was  recorded  for 
experiments  performed  pre-validafion,  during  fhe  CLP  validafion,  and  peripheral  fo  fhe  validation 
in  well-documented  research  studies  and  is  presented  vs.  passage  number  in  Figure  8.  It  was 
observed  that  the  PC  titer  was  reduced  as  the  cell  passage  age  increased.  All  other  assay 
constituents  such  as  medium,  FBA,  agarose,  cell  culturing  practices,  cell  concentration,  etc.,  had  all 
remained  constant,  as  recorded  in  the  GLP-compliant  study  records. 

Figure  8.  Positive  control  data  collected  on  Vero  E6  cells  of  various  passage  ages. 


Vero  E6  cell  passage  number 


Figure  8.  Legend.  Twenfy  eighf  measuremenfs  of  fhe  EBOV  PG  collected  over  pre-,  posf-  and  on- 
validafion  sfudies  were  performed  on  cell  passages  ranging  from  30  fo  41.  Averages  and  sfandard 
deviafions  are  presenfed  for  mulfiple  resulfs  on  fhe  same  passage.  The  solid  line  depicfs  fhe 
nominal  PG  fargef  fifer  of  1.09  x  105  PFU/mL,  and  fhe  dotted  line  denotes  fhe  lower  PG  cufoff  limif 
of  34,469  PFU/mL,  below  which  fhe  assay  fails  and  a  repeal  is  required. 

The  conclusions  from  fhis  compilafion  of  PG  dafa  supporf  fhe  use  of  cells  af  only  passage  36  and 
younger  for  fhe  validafed  EBOV  assay.  The  cufoff  of  36  was  chosen  due  fo  fhe  number  of  dafa 
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points  which  fell  comfortably  above  the  PC  cutoff  limit,  unlike  those  for  passages  37  to  40,  even 
though  some  of  those  values  passed.  The  limitation  of  cell  culture  passage  age  36  as  the  cutoff  for 
EBOV  plaque  assay  methods  may  be  in  contrast  with  the  findings  for  the  validated  MARV  assay,  on 
which  the  other  lot  of  Vero  E6  cells,  was  evaluated  and  not  found  to  have  a  passage  age  limitation 
[8].  Cells  at  passage  ages  younger  than  31  or  32  are  generally  not  available  from  USAMRIID  cell 
culture  due  to  the  laboratory's  receipt  of  the  cell  stock  at  passage  age  24  from  BEI  Resources.  Once 
the  cells  were  received  they  were  amplified  several  passages  and  banked  for  frozen  storage.  Banked 
frozen  cells  are  thawed  and  passaged  in  culture  to  fill  orders,  thus  an  order  for  6-well  plates  can 
typically  be  filled  with  cells  only  as  young  as  passage  31  or  32.  Acceptance  of  the  narrow  window  of 
cell  passage  ages  affects  the  GLP-compliant  USAMRIID  cell  culture  lab,  and  dictates  the  ages  of 
cells  they  may  maintain  in  passage  for  filling  orders  for  validated  studies. 

4.  Discussion 

Different  filovirus  challenge  stocks  can  have  different  provenances  which  may  impact  their  ability 
to  produce  equivalent-looking  plaques  in  a  plaque  assay.  The  studies  in  this  EBOV  validation  were 
completed  with  a  virus  preparation  of  EBOV  Zaire  EBOV-Kikwit,  which  has  been  in  use  for  MCM 
evaluations  at  USAMRIID  [14,15].  The  stock  was  produced  in  cell  culture  (passage  3)  and  is  known 
to  have  a  predominantly  8U  genomic  phenotype  [1].  When  grown  in  vitro  (e.g.  Vero  E6  cells  or 
similar)  EBOV  appears  to  have  a  preferred  8U  genomic  state,  but  a  preferred  7U  genomic  state  in 
vivo  [1,16].  Hence,  a  7U  mRNA  editing  site  is  perhaps  required  for  viral  fitness  in  a  mammalian 
host,  where  it  apparently  makes  and  secretes  a  higher  amount  of  sGP.  Gonversely,  an  8U  genomic 
phenotype  may  have  a  growth  advantage  in  culture.  It  could  enhance  virus  particle  release  in  vitro, 
making  it  the  genomic  phenotype  responsible  for  high  titer  growth  of  virus  stocks  in  vitro  [16,17] 
There  exists  a  7U  Kikwit  stock,  which  was  grown  by  carefully  limiting  cell  culture  passage 
adaptation  of  the  virus,  that  may  be  preferred  as  a  virus  challenge  stock  in  NHP  studies  in  some 
filovirus  research  circles  add  [17,18].  Use  of  either  the  8U  or  the  7U  variants  may  be  justifiable  for 
animal-rule  studies  to  develop  the  NHP  infection  model  itself,  or  to  test  the  efficacy  of  vaccines  or 
therapeutics  [13].  Beyond  the  preparations  of  EBOV-Kikwit  which  are  distinguishable  as  7U  or  8U, 
the  EBOV-Makona  variant  is  also  now  being  amplified  as  a  challenge  stock  reagent  for  animal 
studies,  with  the  intent  of  investigating  the  EBOV  disease  caused  by  the  2014-2015  outbreak  variant 
[19,20].  The  MARV  Angola  variant  is  in  heavy  use  for  MARV  animal  rule  studies  supporting  MGM 
countermeasure  development  [13] ,  but  other  variants  of  MARV,  such  as  Musoke  and  Gi67  could 
also  be  used  for  important  research  [21,22]. 

The  existence  of  these  multiple  variants  of  virus  prepared  as  useful  stocks,  and  the  implications 
each  brings,  has  a  bearing  on  the  validated  filovirus  plaque  assay.  The  EBOV  validation  was 
completed  with  EBOV-Kikwit,  a  lot  which  may  have  the  highest  level  of  cell  culture  adaptation,  if 
indeed  the  8U  genomic  phenotype  is  a  marker  of  in  vitro  production.  This  virus  preparation,  or  lot, 
was  used  to  prepare  the  PG  and  QGs,  and  they  served  as  the  working  test  samples  in  the  EBOV 
validation  assay.  The  assay  was  validated  based  on  the  behavior  of  these  PG  and  QG  samples 
against  reasonable  acceptance  criteria  set  a  posteriori  from  optimization  and  characterization  study 
data.  If  a  different  lot  of  virus,  comprised  of  a  different  variant  or  genomic  phenotype,  is  tested  in 
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either  the  MARV  or  EBOV  validated  assay,  it  may  or  may  not  perform  equivalently  well  in  the 
assay  validated  for  fhe  original  fesf  varianf  or  lof.  The  FDA  guidance  for  biological  assay 
validations  indicates  that  any  change  to  critical  reagents,  such  as  the  PC  or  test  samples,  or  the 
species  of  a  mafrix,  will  likely  require  a  parfial  validafion  [7].  A  parfial  validafion  can  bridge  fhe 
performance  of  fhe  new  variable  fo  fhe  performance  of  fhe  old  in  fhe  known  validafed  mefhod. 
Addifional  virus  varianfs,  stocks  or  lot  preparations,  cell  lines  or  other  major  variables  will  need  to 
be  assessed  for  any  pofenfial  changes  fhaf  fhey  may  bring  fo  fhe  validafed  assay,  and  new  PCs  or 
QCs  may  be  required  for  full  fesfing.  For  example,  fhe  "7U"  preparafion  lof  or  Makona  varianf  may 
have  a  complefely  differenf  plaque  appearance  fhan  fhe  previously  validafed  varianf.  Bridging 
validation  studies  may  be  needed  to  ascertain  whether  the  hypothetical  new  plaque  appearance  can 
be  similarly  quantitated  independently  by  two  analysts  in  detailed  precision  studies  such  as  the 
ones  presented  in  this  work.  Alternatively,  plaques  from  an  unfesfed  lof  may  need  more  or  less 
fime  fo  develop  info  fhe  mosf  robusf  and  counfable  plaques  possible.  Robusfness  and  opfimizafion 
sfudies  may  be  required  before  such  a  bridging  validafion,  fo  demonsfrafe  fhaf  a  cerfain  day  is 
superior  for  counfing.  No  maffer  fhe  genofype  or  provenance,  if  will  be  imporfanf  fo  fesf  new  virus 
lofs  in  characferizafion  and  opfimizafion  sfudies,  before  assuming  fhe  currenf  validafed  assay  will 
perform  opfimally  for  fhe  new  lof. 
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Supplementary  Figure  1. 


The  Bland- Altman  plot  below  is  a  visual  representation  of  the  agreement  of  the  MARV  plaque  counts 
made  by  the  two  analysts  for  each  Precision  experiment.  The  difference  between  counts  of  each  well  are 
plotted  on  the  y-axis,  while  the  mean  of  the  two  counts  for  each  well  are  plotted  on  the  x-axis.  The  central 
reference  line  in  each  plot  is  the  mean  of  the  differences  between  counts;  while  the  two  other  reference 
lines  represent  two  standard  deviations  of  the  differences  above  and  below  the  mean.  It  appears  that  the 
fewer  the  plaques  counted,  the  closer  the  data  fell  to  that  central  reference  line:  therefore,  it  is  possible 
that  when  there  are  fewer  plaques  to  count,  there  is  less  variation  between  two  analysts.  The  Bland- 
Altman  analysis  results  for  MARV  are  very  similar  to  those  seen  for  the  EBOV  analysis. 


Bland-Altman  Plot 


100 

75 

50 

25 


“  0 
c 

3 

O 

e  -25 

O) 

I 

<D 

ja 

S  100 

c 

O) 

§  ^5 


50 

25 

0 

-25 


Experiment  =  1 2-2-1 

Experiment  =  12- 2-2 

•• 

•  • 

••  • 

• 

1 

• 

«  •  •  » 

•  •  ••  •a 

• 

• 

Experiment  =  12-2-3 

Experiment  =1 2- 2-i 

1 

•  • 

f  M 

• 

A  *  .JW  • 

■ 

^  A*.,— ^ 

• 

• 

25  50 


75  100  125  0  25 

Mean  of  Counts 

Sample  •  OC1  •  QC2  >  QC3 


50  75  100  125 


UNCLASSIFIED 


